[KIP-932] : Implement Share consumer interface with poll API by k-raina · Pull Request #2217 · confluentinc/confluent-kafka-python

Kaushik Raina (k-raina) · 2026-03-12T17:24:53Z

Summary

Implements ShareConsumer, a new Kafka client type for KIP-932 share group consumption
Added New file: src/confluent_kafka/src/ShareConsumer.c
ShareConsumerHandle struct inheriting from Handle via first-member embedding — enables safe (Handle *)self casts and reuses all common callback/TLS infrastructure
subscribe(topics) / unsubscribe() / subscription() — topic subscription management via rd_kafka_share_* APIs
consume_batch(timeout=-1) — batch-only consumption with chunked polling for Ctrl+C interruptibility; uses - CallState_begin/end and check_signals_between_chunks for correct GIL and TLS lifecycle management
close() — graceful shutdown: attempts broker close then always destroys handle

Known limitations (TODOs in code)

Item	Blocked On
rd_kafka_set_log_queue()	Needs a specific `rd_kafka_share_set_log_queue()` wrapper to handle the share consumer handle.
OAuth Background Callbacks	Missing `rd_kafka_share_sasl_background_callbacks_enable()` export in the C API.
max_poll_records	Currently hardcoded to 10005; requires updating to the `librdkafka` double-pointer API for dynamic config reads.

Additional Changes

Additional commit bc6101a to build python binding with kip-932 branch for librdkafka. We can remove this commit in future before merging to master.

Provide feedback